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Abstract. The educational potential of robotics kits as a form of control technology will 
remain undervalued until meaningful observation parameters are identified to enable a 
better understanding of children's control strategies. For this reason, this paper aims 
primarily to identify and classify the heuristics spontaneously applied by 6-10 year old 
children interacting with robotic devices containing specific transparency features (i.e. 
programmability) and interactivity features (i.e. immediacy of feedback). Two studies 
are described: an exploratory investigation into the control of a Lynx AL5A arm and a 
pilot study about the control of a Lego Mindstorms NXT®. Two issues relating to control 
heuristics are addressed: the heuristic shift and the perceived and objective level of task 
difficulty. The results demonstrate that three main types of heuristic emerge: (i) 
procedural-oriented, (ii) declarative-oriented, and (iii) metacognitive-oriented. 
Limitations of the difficulty indicators used and shift patterns proposed are discussed in 
relation to future research. 
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Introduction: Identifying cognitive issues in control technology 

Technological kits are longstanding tools in pre-school and school education (Papert & 
Harel, 1991). As a specific kind of technological kit, robotic kits are designed as constructible 
and programmable devices. This means that with the benefit of guidance from educators 
and manuals, children can shape their robot, design its mechanisms and command its 
sensors and actuators. Two inherent features of robotics kits are their transparency, which 
refers to the openly accessible programmability of the robot (Kynigos, 2008; Resnick, Martin, 
Sargent et al., 1996), and their interactivity. Traditionally defined as the set of processes, 
dialogues, and actions through which a human user employs and interacts with a computer 
(Baecker & Buxton, 1987), within the present work interactivity refers to the immediacy of the 
feedback given by the device when a child programs and executes the commands. These two 
features of transparency and interactivity give robotics kits high educational potential 
compared with other kinds of educational robots that are not - or that are only partially - 
constructible and programmable (Gaudiello & Zibetti, in press). 

However, here we argue that this educational potential can only become fully developed 
once we acknowledge the cognitive strategies applied by children when controlling the 
robot. We can call such strategies control heuristics ; these are special kinds of strategies that 
are meant, on the one hand, to modulate the inner hardware/software features of robotics 
kits on the basis of both a task's requirements and of an understanding of the device itself, 
and, on the other, to orient children toward specific categories of knowledge. Thus, this 
paper has two aims. The first aim is to identify children's control heuristics, by observing 
their emergence and assessing their robustness through two different kinds of task and with 
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two different kinds of robotic device. In doing this, we aim to propose a methodological 
contribution to one of the most critical aspects of educational technology: the identification 
of appropriate variables to observe when children approach a new form of technology. This 
identification is crucial for the elaboration of clear interpretations and generalizations about 
the relationship between cognition and technological educational devices (Zuga, 2004). In 
our view, children's control heuristics can be considered as meaningful variables. For this 
reason, we investigate the identification of heuristics as well as the following heuristics- 
related issues: (i) heuristic shift, and ii) the perceived and objective level of task difficulty. 

The second aim is to classify the identified control heuristics by anchoring them to a 
theoretical framework which accounts for the categories of knowledge acquired by children 
when they undertake robotics kit tasks. We believe this important in order to better 
understand how these types of robots could be exploited effectively for educational 
purposes. 

Children’s heuristics of robot control 

Heuristics can be defined as general rules or strategies that guide our actions in a problem¬ 
solving task, and that take into account contextual constraints as well as the final goal of the 
task (Newell & Simon, 1972). In the case of robotic control, heuristics are represented by the 
general rules that guide our actions to control the robot, contextual constraints are 
represented by the inherent features of the robot, and the goal of the task is represented by 
the end-state solution. Control heuristics can thus be used by children to apply those actions 
that modulate the inherent features of the robots in order to achieve the solution to the task. 

According to Anderson and colleagues (Anderson et al., 2001), children make use of 
particular types of cognitive processes when approaching a problem-solving task in a 
educational environment: remembering, which means retrieving knowledge from long-term 
memory; understanding, which is constructing meaning from instructional messages; 
applying, which is carrying out a procedure in a given situation; analyzing, which is 
breaking a whole into parts and apprehend their relation; evaluating, or making judgments 
based on criteria and standards; and creating, by putting elements together to form a 
coherent or functional whole (Anderson et al., 2001, p.68). 

These processes result in four categories of knowledge: factual knowledge, relating to the basic 
elements needed to know about a discipline or to solve problems in it (i.e. knowledge of 
terminology and specific facts); declarative knowledge, concerning the interrelations among 
the basic elements within a larger structure that enable them to function together (i.e. 
concepts, categories, principles and models for this discipline); procedural knowledge, 
constituted of the procedures, techniques, and methods as well as the criteria for using them; 
and metacognitive knowledge, referring to knowledge about task demands, as well as 
strategies and one's ability to accomplish tasks. Together, declarative and conceptual 
knowledge represent knowledge of “what", while procedural and metacognitive knowledge 
represent knowledge of “how to". 

Due to the novelty of robotic kits as educational tools, it is premature to talk about the 
cognitive processes underpinning their use. However, a number of studies have investigated 
the cognitive strategies at work as children gain programming skills during problem solving 
tasks (Ribeiro, Coutinho, & Costa, 2011; Carver & Klahr, 1986; Highfield et al., 2008), as well 
as the scientific and technological knowledge acquired during these tasks (Sullivan, 2008; 
Alimisis 2009). However, in our view, these findings still represent a level of analysis that 
needs to be anchored to a better defined taxonomy, in order to unify the rich experimental 
data from these studies within a reasoned theoretical framework that can account for the 
impact of educational robotics on children's knowledge acquisition. 
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Table 1. The proposed classification system for children's control heuristics inspired by Anderson 

etal. (2001) 


Procedural-oriented 

heuristics 

Declarative-oriented heuristics 

Metacognitive- 
oriented heuristics 

Description 

Task-driven heuristics, 
in which sequences of 
actions are applied by 
children using a trial- 
and-error strategy, 
primarily to achieve the 
solution for the task by 
following an implicit 
procedure. 

Knowledge-driven heuristics, in 
which single actions are applied 
by children through a reasoned 
strategy, mainly to seek explicit 
information about the rules of 
the task. 

Awareness-driven 

heuristics, by which 
actions are applied by 
children through both 
trial and error and 
reasoned strategies, 
mainly to establish the 
limits of the task and 
to assess their own 
understanding of the 
task. 


Moreover, if a considerable amount of literature has been devoted to the interactivity of 
computer interfaces (Rada & Michailidis, 1995; Svanaes, 2000) and of other kinds of 
technological devices (Kennewell et al., 2008; Yacci, 2000; Rose, 1999), no experimental study 
can be found that rely on control heuristics as a means to explore the educational potential 
of the interactivity feature in robotics kits. For this reason, we conducted an exploratory 
study using a robotic device of low transparency and high interactivity as a preliminary 
investigation for a subsequent pilot study, where the device children used during the task 
was characterized by higher transparency and lower interactivity. We aimed to develop a 
classification system for the control heuristics emerging in two different child-robot 
interaction contexts, based on Anderson's taxonomy (Anderson et al., 2001), in order to 
understand which strategies children apply to modulate the interactivity feature 
(transparency being a secondary concern in the present study). We considered Anderson's 
procedural, declarative and metacognitive knowledge categories to be particularly 
indicative of the kinds of knowledge that orient the strategies used by children when they 
control a robot in order to solve a problem (Table 1). The definition of the metacognitive- 
oriented heuristic used here is based on studies within the educational robotics literature 
that have demonstrated that educational robots enhance metacognitive attitudes (cf. Denis & 
Baron, 1993). 

Before presenting the studies, we will discuss two heuristic-related issues, (i) heuristic shift, 
and (ii) the level of objective and perceived task difficulty. 

Heuristic shift 

The issue of shift phases in children's learning is indeed a central one. The reason is simple: 
learning requires change. A considerable variety of conceptual change models can be found 
in the literature (Harrison & Treagust, 2001). According to Sfard (1991), problem-solving 
consists in an intricate interplay between procedural and declarative knowledge. However, 
as Sfard highlights, procedural knowledge is, for most people, the first step. Borrowing from 
Piaget, who states that a process has been interiorized if it can be carried out through mental 
representations (Piaget, 1972, p.14). Sfard explains that the transition from procedural to 
declarative knowledge is a long and difficult process, accomplished through the 
interiorization of procedures and their consequent reification in explicit definitions. A 
further shift is needed to pass from procedural and declarative to metacognitive knowledge, 
since the latter implies a higher level of knowledge: awareness of one's knowledge. 
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Although this shift is considered to occur progressively in traditional educational settings, 
new educational settings such as educational robotics could demonstrate that it does not 
necessarily follow the same progression when children are confronted with technology with 
inherent hardware/software features that elicit primarily one type of knowledge rather than 
another, i.e. metacognitive knowledge in the case of robotics kit technology. In our opinion, 
it would thus be relevant to ascertain whether children persistently adopt one type of 
heuristic or if they shift between different types when repeating the same task twice. In 
particular, it would be pertinent to observe in which order this shift occurs, and whether it is 
progressive (i.e. a sequential use of control heuristics oriented to procedural knowledge, 
then to declarative knowledge, and finally to metacognitive) or non-progressive (a non- 
gradual use of control heuristics, e.g. from procedural to metacognitive). In relation to the 
literature reported previously, a progressive shift could be considered as an indication that 
the impact of the robotic learning environment on children's knowledge acquisition is not 
different from other traditional learning environments; in contrast, a non-progressive shift 
could indicate that the inherent features of educational robots produce specific control 
heuristics that elicit particular types of knowledge acquisition. 

Perceived task difficulty 

Task difficulty is a composite factor which plays an important role in children's mastery of 
educational technology. Established methods within Psychology of evaluating technology 
usability traditionally relies on quantitative indicators to assess the difficulty of the task, 
such as the number of errors, completion time, etc. These indicators are also usually 
employed in what has been named the Psychology of programming (Pea & Kurland, 1984). 
Diverse qualitative indicators have been introduced to educational robotics, ranging from 
the number of blocks making up the program (Caci, Cardaci & Lund, 2003), to the type of 
conditional rule that determines the adaptive behavior of the robot in an environment. This 
heterogeneity of indicators demonstrates that it is not clear how the objective difficulty of 
the task and the perceived difficulty of the task can be distinguished in robotics-based 
activities. 

An interesting proposal in relation to this was made by Levy & Mioduser (2008). According 
to these authors, when the task to be accomplished in interaction with the robot is perceived 
as easy, children tend to assume a technological perspective, that is, they employ a more 
engineering-related vocabulary in the spontaneous or enhanced questions and remarks that 
they address to peers and educators. The higher the level of difficulty of the task, the more 
the children shift to a psychological perspective, that is they use a more anthropomorphic 
vocabulary; for instance, they attribute states to the robot that are normally attributed to 
humans, such as intentions, volitions, fears, wishes, etc. Therefore, in order to capture both 
the objective and the perceived levels of task difficulty, we chose to measure both the 
number of errors made by children during programming and the type of engineering vs. 
anthropomorphic vocabulary employed during group interaction. 

Identifying control heuristics for a Lynx AL5A robotic arm 

In the first stage of our research, we conducted an exploratory study into children's control 
heuristics during the Lirst Lego League® competition taking place at Drancy (a suburb of 
Paris) during winter 2011. In addition to the main competition program, various educational 
stands were set up for the purpose of testing several robotic devices, with educators setting 
micro-challenges to the Lirst Lego League® participants and to children attending as visitors. 
Our exploratory study was realized in this context: we observed actions sequences used by 
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children controlling a basic model Lynx AL5A2 robotic arm in one of these micro-challenges, 
and we classified into heuristics in order to test the relevance of our approach and our 
proposed classification system for general control heuristics. 

Context, robotic devices and children’s tasks 

Our stand offered a simple activity with a Lynx AL5A robotic arm (Fig.la), composed of a 
shoulder fixed to a table, a forearm, an arm, a wrist, a hand-gripper, an Arduino card, a 
motor for each of the five components and a knob for each motor (Fig.lb). Volunteers 
amongst the children at the competition or from the public could manipulate the robotic arm 
in order to grasp, lift and drop a sugar lump into a coffee cup. No programming was done: 
commands were given by manipulating the knobs. This kind of robotic device is thus not 
characterized by a high level of transparency, as its control merely requires the use of knobs 
to instruct the arm to move in four directions (up, down, left, right) and to open/ close the 
hand gripper. In contrast, the fact that children can observe the effects of their commands in 
real time when manipulating the knobs engenders a high level of interactivity. Forty 
children (6-10 years old) volunteered for this activity during the all-day competition. Each 
trial for each child lasted 10 minutes. We were able to observe and classify children's control 
heuristics over the forty trials. 

Data collection and coding 

In order to observe if control heuristics were spontaneously deployed by children with this 
kind of robotic device and task, we recorded on an observational grid the nature of the 
actions performed by each child when he/she controlled the robot. We then coded the 
actions and defined the action sequences in terms of the types of heuristics outlined in Table 
1. For this task and the Lynx AL5A robotic device, the action sequences were thus 
operationalized in terms of the three heuristics described in Table 2. The type of heuristic 
applied by a child was identified by two judges according to the type of action sequences 
he/she applied to the knobs. 

Observed heuristics and outcomes 

Three emergent heuristics were observed among the children. 

(i) 60% of the children used the knobs to achieve a sequence of task-related commands for 
grasping, lifting and dropping an object. These children were mainly interested in applying a 
procedure in order to achieve the end-solution state, sometimes proceeding by trial-and- 
error strategies. 



Figure 1. (a) The Lynx AL5A robotic arm being controlled by children during the exploratory study 
at the First ® League, (b) A child manipulating one of the five knobs corresponding to one of the 
five motors for the arm in order to pick up a sugar lump and put it in the coffee cup. 
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Table 2. Classification of the action sequences in terms of control heuristics used to control the 

Lynx AL5A 


Procedural-oriented 

heuristics 

Declarative-oriented 

heuristics 

Metacognitive-oriented 

heuristics 

Action 

sequences 

Children manipulated 
the knobs one after 
another, observing the 
corresponding motor 
activation and 
adjusting the 
movement in real time, 
in order to solve the 
task. 

Children manipulated each 
of the five knobs and 
observed the corresponding 
motor activation. They only 
started to solve the task 
once they had understood 
the one-to- one 
correspondence between 
each knob and the related 

motor movement. 

Children manipulated the 
knobs or a combination of 
knobs, observing the 
corresponding movement 
(i.e. the activation of more 
motors) in order to assess 
what they could and could 
not do with the robot. They 
only started to solve the 
task or proposed different 
or further tasks after this. 


(ii) 30% of the children used the knobs in order, first to understand each knob's function (i.e. 
the one-to-one correspondence between knob manipulation and motor movement) and then 
to accomplish the task. These children were more interested in acquiring knowledge about 
task-related commands, in order to apply a reasoned (i.e. not trial-and-error) procedure 
afterwards. 

(iii) Only 10% of the children used the knobs in order to explore the general functioning of 
the robotic arm. These children were more interested in becoming aware of what they could 
do and could not do, in terms of possibilities (e.g. combining different knobs to obtain 
specific robotic arm movements) and limitations (e.g. checking the maximum rotation of a 
motor beyond which the robotic arm blocked); thus they did not restrict their attention to 
task-related commands. 

These first results showed that the majority of children seemed to pay more attention to 
achieving the solution and to acquiring knowledge about the functioning of the task-related 
commands, rather than to gaining a general awareness of the possibilities and limitations 
offered by the robotic device. This is coherent with those studies that show that trial-and- 
error strategies are predominant when children approach a new technology (Ribeiro et ah, 
2011) and that they relate to procedural knowledge (Anderson et al., 2001; Sfard, 1991). 
Moreover, the emergence of overt and diverse types of heuristics enabled an initial 
validation of our approach, which consisted in recording and classifying the nature of action 
sequences performed by the children (Table 2). Using Anderson's taxonomy (2001), we were 
able to link these action sequences to the type of knowledge acquisition favored by the use 
of such heuristics when children controlled the robotic device (Table 1). This coding 
procedure and the initial results encouraged us to investigate children's control heuristics 
further in a later study on a different task and with a different robot (Lego Mindstorms 
NXT®). 

Pilot study: control heuristics for Lego Mindstorms NXT® kits 

Our pilot study was designed to verify, as a first aim, whether the proposed classification 
(Table 1) was also relevant in a different device and problem-solving context. For this reason 
we used a different kind of robotic kit. Lego Mindstorms NXT®, and a different kind of 
problem-solving task. The Lego robotic kit includes a programming interface with a graphic 
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code: drag and drop icons are used to program sensors and actuators. Compared with the 
Lynx AL5A robotic arm. Lego Mindstorms NXT® has a higher level of transparency: all the 
sensors and the actuator can be programmed using dedicated icons with stylized symbols 
on them (e.g. a stylized wheel on the motor programming icon, a stylized sun on the light 
sensor programming icon) and by setting the parameters of these icons (e.g. the speed of the 
motor or intensity of the light), so that children do not have to guess which icon corresponds 
to each command, nor what the effect of such a command might be. Moreover, Lego 
Mindstorms NXT® has a lower level of interactivity than the Lynx AL5A robotic arm. 
Moreover, in the pilot study we were able to better control experimental procedure so to 
investigate the two issues of heuristic shift and perceived task difficulty as a second aim. In 
order to meet these aims, we refined the definition of the three heuristics identified in the 
exploratory study by adapting them to the specific Lego robotic device and task. The three 
heuristics were thus operationalized as outlined in Table 3. Finally, we wanted to assess if 
the proposed task was perceived as easy or difficult, and if the proposed indicators (number 
of errors and type of vocabulary) were appropriate to test the level of task difficulty and 
perceived task difficulty respectively. 

Hypothesis and Expectations 

We expected children to demonstrate at least three types of programming heuristics when 
controlling the robot (Table 3): (i) Procedural-oriented; (ii) Declarative-oriented and (iii) 
Metacognitive-oriented. Other heuristics, specific to the robotics device, could possibly 
emerge. In particular, since trial-and-error strategies have been observed to predominate 
when children approach a new technology (Ribeiro et al., 2011) and since these strategies 
typically relate to procedural knowledge as the first kind of knowledge to be acquired 
(Anderson et al., 2001; Sfard, 1991), we expected procedural-oriented heuristics to occur 
more frequently than the others. 

In relation to heuristic shift, because several studies have highlighted the role of Lego 
robotics kits in enhancing metacognitve attitudes (see Danis & Baron, 1993), we expected a 
non-progressive shift between different control heuristics; that is, we expected children to 
pass from procedural to metacognitive-oriented heuristics, without passing through 
declarative-oriented heuristics, or to start directly with metacognitive oriented heuristics. In 
relation to task difficulty, we expected children using procedural oriented heuristics making 
more errors because this kind of heuristic mostly implies trial and errors procedures. 


Table 3. Classification of action sequences in terms of control heuristics used to control the Lego 

Mindstorms NXT® 


Procedural-oriented 

heuristics 

Declarative-oriented 

heuristics 

Metacognitive- 
oriented heuristics 

Action sequences Children drag and drop 
several programming 
icons onto the software 
workspace and they 
execute the whole 
sequence of icons (i.e. 
the program). 

Children drag and drop a 
programming icon onto 
the workspace and they 
execute it individually 
before adding the 
subsequent programming 
icons needed to complete 
the program. 

Children drag and drop 
one or more ? 
programming icons ? 
onto the workspace, 
they set the parameters 
and they execute the 
program more times, 
trying different 
parameters. 
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Figure 2. (a) The materials used in the pilot study: colored objects (blue, red, and yellow paper), 
objects with different temperatures (an ice cube, a glass of hot water and a glass of room 
temperature water), and objects producing different sounds (a bell, a whistle and a percussion 
stick), (b) Children testing a temperature sensor on a plastic ice cube. 

Concerning perceived task difficulty, as recent studies in robot-child interaction have shown 
that children who find a task difficult use anthropomorphic language while children who 
find it easy use technological language (Levy & Mioduser 2008), we expected children using 
more anthropomorphic terms because they had never used robotics kits before, so some 
difficulties could emerge in their first approach to this technology. 

Participants and materials 

Twenty-six children (6-10 years old) participated in the study (17 boys and 9 girls). Thirteen 
participants were aged 6-7 years and thirteen participants were aged 8-10 years. Three 
children worked on each robotic device (a Lego® Mindstorms NXT kit including light, sound 
and temperature sensors, and a computer provided with Lego software). Children of 
different ages were grouped in order to balance age differences. Each group was 
accompanied by an educator who guided the activity and an experimenter who took notes 
on an observational grid without intervening in the activity. No child had used a Lego robot 
before. The task consisted in making the robot's actuators (i.e. the motor) react when a 
defined intensity of light, sound, and temperature was detected by the sensors. To achieve 
this aim, children were first taught to use the NXT iconic language to calibrate the sensors 
(i.e. to define a threshold beyond which the robot had to react). Pencils and papers were 
distributed to enable children to note values detected by the sensors. The educational 
activity presented to the children was designed to portray the robot as an inquiry tool to 
investigate the properties of light, sound and temperature, using an introductory analogy 
between human senso-motricity and the robot's sensors and actuators. The scenario 
(Gaudiello, Zibetti & Pinaud, 2012) was specifically improved and adapted within a joint 
frame including the Carrefour Numerique educational seminars at the Cite des Sciences et 
de T Industrie in Paris, and the Pri-Sci-Net European Project. The aim of the Carrefour 
Numerique educational seminars is to disseminate scientific and technological information 
to a broad public. The objective of the Pri-Sci-Net European Project is to implement Inquiry 
Based Learning (IBL) for Scientific Education in primary schools. 

Tasks, procedure and data collection 

The experiment included an introductory phase, in which educators engaged children in a 
discussion about the differences between automats and robots by showing them images 
selected by the experimenters. This phase was designed to investigate the children's prior 
knowledge of robotics and to provide a general understanding of the robot as a particular 
type of machine, which can be programmed to sense and act in order to accomplish multiple 
tasks, depending on the current state of the surroundings. A second phase followed in 
which children were prompted to form groups, with each group choosing the kind of sensor 
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(light, sound, or temperature) they wanted to program. Initially, the educator demonstrated 
how to use the programs already stored in the intelligent brick. Subsequently he/she 
outlined the functioning of the NXT icon programming language: the meaning of the 
programming icons, the parameters, how to create a basic program by joining several 
programming icons, how to download and execute the program. Then the children were 
introduced to the target problem: how to make the robot react to information detected by the 
sensor. For the light sensor group, the target was to make the robot go forward if it detected 
a green light and to go backwards if it detected a red light. For the sound sensor group, the 
target was to make the robot say "shhh!" if the detected sound exceeded a defined 
threshold. For the temperature group, the target was to make the robot say "hot" if the 
temperature sensor detected a temperature value exceeding 40° degrees. Each child within 
the group was encouraged to complete a programming trial. Afterwards, the children were 
invited to program the sensor to solve the target problem. Once the task had been 
completed, the children could have further trials or ask for a different sensor to program. 
Task execution for each group lasted on average 60 minutes. Each group completed on 
average 6 trials on different sensors (on average 2 for each child). The problem-solving 
phase lasted on average 30 minutes. Finally, all groups were collectively invited to discuss 
their views on the activity and their suggestions for how future activities could be improved. 

Behavioral data were collected by the experimenters throughout the activity. Actions, errors 
and verbalizations between children as well as between children and educators were 
collected during the programming phase using an observational grid for each child in each 
group. A colored label was placed on each child's hand, in order for the experimenters to be 
able to identify the participants on the observational grid. The educators were instructed to 
refer to the children using their assigned color when interacting verbally. Screen capture 
software (CamStudio) was used to register cursor actions on the screen,the order in which 
actions occurred when children programmed the robot as well as verbalizations. This 
information was used only to confirm descriptive analysis or to complete missing data. The 
type of heuristic applied by a child was identified by two judges according to the type of 
action sequences (Table 3) he/ she applied during the programming. Errors were coded by 
counting each time a child used an incorrect icon or an incorrect icon sequence. Language 
was coded by noting when children referred to the robot using anthropomorphic terms (e.g. 
verbs indicating intentions, worries or other psychological states normally attributed to 
humans) and engineering terms (e.g. names of specific kit components or explanations of 
procedures and terms using technical vocabulary, such as cable, sensors, computer-robot 
relations, etc.). 

Results and interpretation 

Twenty-one of twenty-six children programmed the robot - with the remaining five children 
tending to participate more in terms of manipulating hardware (e.g. connecting sensors to 
the processor) or proposing and evaluating solutions. Forty-two programming trials were 
carried out during the study (mean: 2 trials per child; 6 trials per group). Only a preliminary, 
exploratory descriptive data analysis was performed. 

Which heuristics are spontaneously developed by children engaged in the control of 
Lego Mindstroms® robotic kits? 

Across the 42 trials we observed a total of 37 heuristics. Overall, our results show that all 
three heuristics were observed (Table. 3). Across the 42 trials, procedural-oriented heuristics 
were observed most frequently (49%), followed by metacognitive heuristics (30%) and 
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declarative heuristics (21%). In line with our expectations, procedural heuristics were 
applied most often by children, i.e. they represented almost half the sampled data. Thus, 
procedural-oriented heuristics appear to represent the most frequently kind of action 
sequence adopted by children approaching this kind of problem-solving task. This kind of 
heuristic typically involves trial and error strategies. Furthermore, the distribution of the 
three heuristics and the predominance of procedural heuristics are consistent with most 
findings in the literature: procedural heuristics, which are mainly task-driven, orient 
children towards the acquisition of procedural knowledge, which is known to precede 
declarative ("knowing what to do to execute a task") and meta-cognitive ("knowing about 
knowing") knowledge acquisition (Sfard, 1991; Piaget, 1972). In contrast with previous 
findings, we observed more metacognitive than declarative-oriented heuristics across all 
trials. There are two possible ways to interpret this result: in terms of the curiosity 
stimulated by the novelty of the device, or in terms of Lego robotics kits' reputed key role in 
enhancing metacognitive attitudes. The novelty effect constitutes a bias often encountered in 
studies involving the impact of new technologies on children's learning attitudes. Further 
analysis of heuristic shift should enable greater understanding of these results. 

How do children shift from one heuristic to another? 

In order to answer this question, we considered the type of heuristic used by each child in 
the first and second trials as the unit of analysis (Table 4). Among the twenty-one children 
who programmed the robot, 38% used only one type of heuristic in the two trials, while 62% 
(thirteen out of twenty-one children) used more than one type. However among this last 
group, a majority of children made non-progressive heuristic shifts, that is they started with 
procedural-oriented heuristics and directly shifted to metacognitive (Table 4). 

Although two trials per child represent a limited data set from which to draw rigorous 
conclusions, the data trend shows that procedural-oriented heuristics predominated, non¬ 
progressive shifts occurred more frequently than progressive shifts, and metacognitive- 
oriented heuristics were applied more extensively than declarative-oriented heuristics. 
However, the surprisingly high occurrence of metacognitive-oriented heuristics still requires 
an explanation. In particular, if the frequent occurrence of metacognitive-oriented heuristics 
is due to the novelty effect of the robotic device, then we would expect this kind of heuristic 
to be used more frequently as the initial or sole approach to the robot. If, in contrast, the use 
of metacognitive-oriented heuristics is specifically triggered by the particular kind of robotic 
device, then we would expect metacognitive heuristics to be used later on, after procedural 
or declarative-oriented heuristics. As the descriptive analysis shows, only one child used 
metacognitive-oriented heuristics as the initial and sole approach, while the other children 
seemed to apply it later on, through progressive or non-progressive shift. In this sense, the 
study seems to confirm claims within the literature about the potential of educational robots 
to enhance metacognitive attitudes. Further analysis using larger sets of programming trials 
is planned to reinforce these initial findings. 


Table 4. Percentage of progressive and non-progressive heuristic shifts between the two trials 
when children controlled a Lego Mindstorms NXT 


Use of single heuristics (no shift) 38% 

(n = 8 children) 

Progressive heuristic shifts 24% 
(n = 5 children) 

Non - Progressive 
heuristic shifts 38% 

(n = 8 children) 

Procedural 

Declarative Metacognitive 

Procedural/ 

Declarative 

Declarative/ 

Metacognitive 

Procedural/ 

Metacognitive 

28% 

5% 5% 

16% 

8% 

38% 
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Is the problem perceived as easy or difficult? 

In order to investigate how much the children understood about a kit they had never used 
before, we decided to break the difficulty factor down into two indicators. The first indicator 
related to children's performance in terms of the number of errors made during problem¬ 
solving. We considered any incorrect programming icon or sequence of icons to be an error. 

The second indicator related to children's verbalizations (the use of engineering vocabulary 
when the task was perceived as easy and anthropomorphic vocabulary when the task was 
perceived as difficult). 

Performance - number of errors: Over the 42 trials (84 blocks, each trial requiring two 
blocks) and across the 37 heuristics, we observed a total of 27 errors during programming. 
52% of errors occurred with procedural-oriented heuristics, 33% with metacognitive- 
oriented heuristics and 15% with declarative-oriented heuristics. This result, partially 
coherent with our hypothesis can be attributed to the fact that not only procedural but also 
metacognitive-oriented heuristics basically involves a trial-and-error approach, with the 
former heuristic leading to different trials being carried out in order to attain the task goal 
and the latter used to understand the robot's possibilities and limitations. In contrast, 
declarative-oriented heuristics did not lead to frequent errors, since these heuristics are 
applied in order to achieve an exact understanding of each task-related command, so that 
errors can be identified step-by-step rather than sought after the entire sequence has been 
executed. 

Language - type of vocabulary: We registered a total of 58 occurrences of both engineering 
markers (e.g. "the robot is programmed to", "the computer tells the robot to do that", "it 
does not work because the cable is not plugged-in", etc.) and anthropomorphic markers (e.g. 
"the robot wants" or "he does not want" and "he is scared", etc.). We did not register any 
specific language marker for six children and for one child we registered 8 markers. The 
remaining nineteen children used between 2 and 4 markers on average. Across the 58 
markers recorded across all groups of children, 29% were anthropological markers and 71% 
were engineering markers. Contrary to our hypothesis, the predominance of engineering 
markers might suggest that the task was perceived as easy. 

General discussion and conclusions 

To summarize, the observations and measures carried out during the Lynx AL5A robotic 
arm study and the Lego Mindstorms NXT® pilot study allow us to draw the following 
conclusions. Classifying and analyzing the action sequences performed by children during 
their interactions with robotic devices within a control heuristics typology and defining the 
corresponding type of knowledge gain can be an insightful way to highlight the educational 
benefits of such devices within a learning framework. Across the two different kinds of 
robotics device and task, three types of heuristics emerged: procedural-oriented, declarative- 
oriented and metacognitive-oriented. Specifically, procedural-oriented heuristics occurred 
more frequently in both studies. Furthermore, metacognitive-oriented heuristics occurred 
more frequently than declarative-oriented heuristics with the Lego robot. The extensive use 
of metacognitive-oriented heuristics and the non-progressive heuristic shift observed when 
children controlled Lego robots seem to confirm evidence in the literature that this kind of 
technological device should increase metacognitive attitudes during knowledge acquisition. 
However, more in depth analysis is needed to ensure that the exploration of the robot's 
general functioning (i.e. the metacognitive-oriented heuristic) is not attributable to the use of 
a specific heuristic, but merely to the device's novelty effect; that is, to the curiosity elicited 
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in children during their first encounter with the robot. Moreover, two trials per child do not 
constitute a sufficiently representative data set to enable us to assess whether a shift from 
one heuristic category to another has actually taken place. Finally, we must consider 
possible interaction effects between errors and adopted heuristics: children might have 
shifted from one heuristic to another in order to counteract errors they had committed rather 
than to voluntarily change their approach. 

The proposed methodology based on a combination of control heuristics and language 
marker and error number indicators has encouraged us to pursue this type of analysis and 
refine these indicators in order to investigate child-robot interaction further; however, with 
relation to level of perceived task difficulty, the surprisingly high use of technological 
language observed among children leads us to interpret these results with a certain degree 
of caution. The fact that children today are widely exposed to technological vocabulary may 
bias an attempt to fully exploit traditional linguistic markers as indicators; traditional 
language markers need to be updated in order to reflect and control for the language 
evolution affecting the future generation of "digital native" children. 

Limitations of the pilot study 

First, although this study mainly addressed the use of control heuristics in relation to the 
low level of interactivity that characterizes Lego, we need to acknowledge the fact that 
Lego's high transparency can also affect children's heuristics. Thus, if on the one hand low 
interactivity may prompt children to exploit declarative heuristics that can reduce feedback 
time, high transparency may on the other hand prompt children to pass straight to 
metacognitive heuristics in order to fully explore the robot. 

Second, the three different sensors included in the Lego robotics kit might have presented 
different levels of difficulty. For example, the temperature sensor displayed the registered 
temperature for several minutes, such that it was not clear that repeated measurements 
needed to be taken in order to calibrate the sensor. 

Third, although we tried to balance age differences across the groups of children, we did not 
control for the level of logical skills, scientific knowledge or technological fluency of each 
child allocated to a group. 

Fourth, the present analysis of sample data did not take into account group dynamics (e.g. 
collaboration, competition, leader-attitude etc.) that might have promoted or impeded task 
completion and knowledge acquisition.y determined by group dynamics or might be related 
simply to the children's age. 

Fifth, different scaffolding styles were applied by different educators within each group, 
with some educators showing a preference for strongly structured guidance and others 
being more open to children's own initiatives. 

Sixth, we only performed descriptive analysis, since the data set (in terms of the number of 
trials per child) was too limited to enable us to generalize our initial results using statistical 
analysis. Therefore, these initial outcomes need to be intepreted with prudence, since at this 
stage we are unable to state that the observed differences are statistically significant. 
Moreover, we need to replicate this experiment to enable better control of children's profiles, 
group dynamics and number of trials. 

Future perspectives 

Overall, the results of the present investigation support the need for further studies to be 
conducted in order to develop several of the proposed issues, provide more detailed results 
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and answer the questions raised in this pilot study. An extended set of trials is required and a 
rigorous experimental design needed to measure the effects of age, of different levels of 
scientific, logical and technological knowledge and skills, and of group dynamics. To 
achieve this, children could be pre-tested and post-tested using questionnaires delivered 
before and after the robotics activity. Finally, more pertinent indicators of perceived task 
difficulty should be specified and stronger distinctions between task difficulty and device 
difficulty introduced. In summary, this study has sought to provide two contributions to the 
field of Educational Robotics: a methodological contribution in terms of the identification of 
meaningful variables in child-robot interactions (i.e. control heuristics and knowledge- 
related language), and the elaboration of a framework to be developed and implemented to 
assess the impact of robotics kits on children's learning. 
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